Predicting and Improving Performance on Introductory Programming Courses (CS1)

This thesis describes a longitudinal study on factors which predict academic success in introductory programming at undergraduate level, including the development of these factors into a fully automated web based system (which predicts students who are at risk of not succeeding early in the introductory programming module) and interventions to address attrition rates on introductory programming courses (CS1). Numerous studies have developed models for predicting success in CS1, however there is little evidence on their ability to generalise or on their use beyond early investigations. In addition, they are seldom followed up with interventions, after struggling students have been identified. The approach overcomes this by providing a web-based real time system, with a prediction model at its core that has been longitudinally developed and revalidated, with recommendations for interventions which educators could implement to support struggling students that have been identified. This thesis makes five fundamental contributions. The first is a revalidation of a prediction model named PreSS. The second contribution is the development of a web-based, real time implementation of the PreSS model, named PreSS#. The third contribution is a large longitudinal, multi-variate, multi-institutional study identifying predictors of performance and analysing machine learning techniques (including deep learning and convolutional neural networks) to further develop the PreSS model. This resulted in a prediction model with approximately 71% accuracy, and over 80% sensitivity, using data from 11 institutions with a sample size of 692 students. The fourth contribution is a study on insights on gender differences in CS1; identifying psychological, background, and performance differences between male and female students to better inform the prediction model and the interventions. The final, fifth contribution, is the development of two interventions that can be implemented early in CS1, once identified by PreSS# to potentially improve student outcomes. The work described in this thesis builds substantially on earlier work, providing valid and reliable insights on gender differences, potential interventions to improve performance and an unsurpassed, generalizable prediction model, developed into a real time web-based system

CSLINC - Development of a National Outreach VLE

Over the last year an online learning platform has been developed and piloted to the Irish second level education system allowing both students and teachers to participate in introductory computing modules. This poster will outline the development of the registration process of a system that is capable of managing potentially 728 schools, 1000+ classrooms and one million students (the entire Irish second level school system). CSLINC is an online student virtual learning environment for computing consisting of several modules built by academics and industry leaders and disseminated to schools through Moodle, our selected virtual learning environment. While Moodle has a certain amount of automation and user management built-in, this poster will present the initial design considerations and the automation process developed to allow for school centered mass registration on Moodle. This is of value to other CER educators who may consider developing such a system and enrollment process. Future work will consist of a detailed publication on the development process

The Elusive Metrics - Are We Telling the Full Story in Educational Data Mining?

The use of Education Data Mining (EDM) has seen a significant increase in recent years. A recent report identified notable concerns with the literature relating to the lack of metrics presented in EDM research (in particular, predicting student performance). This poster presents details on these concerns that may inhibit future re-validation studies or worse, models that initially report strong findings which may not generalise. This poster also declares a call to action for future studies to present such metrics, and finally describes ongoing work in this space (a systematic literature review

CS1: how will they do? How can we help? A decade of research and practice

Background and Context: Computer Science attrition rates (in the western world) are very concerning, with a large number of students failing to progress each year. It is well acknowledged that a significant factor of this attrition, is the students’ difficulty to master the introductory programming module, often referred to as CS1. Objective: The objective of this article is to describe the evolution of a prediction model named PreSS (Predict Student Success) over a 13-year period (2005–2018). Method: This article ties together, the PreSS prediction model; pilot studies; a longitudinal, multi-institutional re-validation and replication study; improvements to the model since its inception; and interventions to reduce attrition rates. Findings: The outcome of this body of work is an end-to-end real-time web-based tool (PreSS#), which can predict student success early in an introductory programming module (CS1), with an accuracy of 71%. This tool is enhanced with interventions that were developed in conjunction with PreSS#, which improved student performance in CS1. Implications: This work contributes significantly to the computer science education (CSEd) community and the ITiCSE 2015 working group’s call (in particular the second grand challenge), by re-validating and developing further the original PreSS model, 13 years after it was developed, on a modern, disparate, multi-institutional data set

Developing an Open-Book Online Exam for Final Year Students

Like many others, our institution had to adapt our traditional proctored, written examinations to open-book online variants due to the COVID-19 pandemic. This paper describes the process applied to develop open-book online exams for final year (undergraduate) students studying Applied Machine Learning and Applied Artificial Intelligence and Deep Learning courses as part of a four-year BSc in Computer Science. We also present processes used to validate the examinations as well as plagiarism detection methods implemented. Findings from this study highlight positive effects of using open-book online exams, with 85% of students reporting that they either prefer online open-book examinations or have no preference between traditional and open-book exams. There were no statistically significant differences reported comparing the exam results of student cohorts who took the open-book online examination, compared to previous cohorts who sat traditional exams. These results are of value to the CSEd community for three reasons. First, it outlines a methodology for developing online open-book exams (including publishing the open-book online exam papers as samples). Second, it provides approaches for deterring plagiarism and implementing plagiarism detection for open-book exams. Finally, we present feedback from students which may be used to guide future online open-book exam development

A Collaborative Online Micro: Bit K-12 Teacher PD Workshop

This poster describes the use of online technology to deliver K12 teacher professional development (PD) during the COVID-19 pandemic in Ireland. Traditionally these sessions are delivered in person, with a focus on hand-on activities, but the sudden changes faced by the closures in Ireland required an alternative approach for delivering these sessions. The PD session presented in this poster was a more technically challenging micro:bit workshop, which was delivered online using the micro:bit classroom. This is typically used as an in-class, one to many instructor tool, and trialing this as a PD collaborative tool, was a novel approach. This poster presents the delivery and methodology of the session, the collaborative online format, and feedback from the participants

CSLINC: A Nationwide CS MOOC for Second-level Students

This poster introduces CSLINC, a free scaffolded MOOC framework tailored to second-level students in Ireland that consists of: an online platform built for accessibility; a suite of modules developed upon international best practices with varying co-creators; and automated assessment and certificates of completion. Its aim is to provide content to promote national CS curricula to all second-level students in Ireland. In September 2021, CSLINC launched to 10,000 students across 100 schools. Future work will include collecting and collating research to validate CSLINC’s goals, scaffolding that will build foundations for national curriculum learning outcomes, and measure its impact on students, their perceptions and follow on CS uptake at second-level in Ireland.https://arrow.tudublin.ie/cddpos/1012/thumbnail.jp

Using Machine Learning Techniques to Predict Introductory Programming Performance

Learning to program is difficult and can result in high drop out and failure rates. Numerous research studies have attempted to determine the factors that influence programming success and to develop suitable prediction models. The models built tend to be statistical, with linear regression the most common technique used. Over a three year period a multi-institutional, multivariate study was performed to determine factors that influence programming success. In this paper an investigation of six machine learning algorithms for predicting programming success, using the predetermined factors, is described. Naïve Bayes was found to have the highest prediction accuracy. However, no significant statistical differences were found between the accuracy of this algorithm and logistic regression, SMO (support vector machine), back propagation (artificial neural network) and C4.5 (decision tree). The paper concludes with a recent epilogue study that re-validates the factors and the performance of the naïve Bayes model

The European Commission and AI: Guidelines, Acts and Plans Impacting the Teaching Of AI and Teaching With AI

Recent developments, guidelines, and acts by the European Commission have started to frame policy for AI and related areas such as ML and data, not only for the broader community, but in the context of education specifically. This poster presents a succinct overview of these developments. Specifically, we look to bring together all publications that might impact the teaching of AI (for example, teacher expectations in the coming years around AI competencies) and publications that affect the use of AI in the classroom. We mean using tools and systems that incorporate both ‘Good Old Fashioned’ AI and those that can directly impact students. This poster is of value to both the European and the wider CER communities and practitioners, as it brings together several guidelines, acts, and plans that are not easily searchable or linked. The publications presented in this poster will impact the teaching of AI and teaching with AI in Europe, and insights can be drawn and compared for other jurisdictions as the educational world adapts to and with AI.https://arrow.tudublin.ie/cddpos/1025/thumbnail.jp

Vulnerable Customers\u27 Perception of Corporate Social Responsibility in the Banking Sector in a Post-Crisis Context

This research aims to determine to what extent corporate social responsibility (CSR) actions developed by bank entities in Spain improve the vulnerable customers\u27 emotions and quality perception of the banking service. Consequently, this increases the quality of their relationship regarding satisfaction, trust and engagement